System and method for annotating and capturing chart data

ABSTRACT

A program is disclosed for permitting a user to directly annotate and capture data associated with a Java™ based applet application running within a conventional browser program. The program addresses sand-box local security restrictions otherwise attendant with such applet by exploiting non-local system resources that are accessible to such types of applications.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a divisional application of Ser. No.09/172,191 filed Oct. 13, 1998 entitled System And Method For Permittinga Software Routine Having Restricted Local Access to Utilize RemoteResources to Generate Locally Usable Data Structure, Which is herebyincorporated by reference.

CROSS REFERENCE TO MICROFICHE APPENDIX

Appendix A, which forms part of this disclosure and is incorporated byreference herein, is a microfiche appendix consisting of 1 sheet ofmicrofiche having a total of 58 frames. Microfiche Appendix A is a listof computer programs and related data in an embodiment of the presentinvention, which is described more completely below.

FIELD OF THE INVENTION

The invention relates generally to a computer software routine whichfacilitates and expands the functionality of secure program routinesrunning on a local client computing system by permitting such routinesto interact with resources located at a remote server. The invention hasparticular use in increasing the utility of routines embodied in Java®applets, which, while running locally on a user's system, cannonetheless be configured to access resources from a remote server so asto circumvent security mechanisms otherwise prohibiting such appletsfrom generating or reading local data structures.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the xerographic reproduction by anyone of the patentdocument or the patent disclosure in exactly the form it appears in thePatent and Trademark Office patent file or records, but otherwisereserves all copyright rights whatsoever.

BACKGROUND OF THE INVENTION

The use of the world wide web (WWW) is increasing rapidly, and so ofcourse is the demand for intelligent systems and software which canpermit users to better and more easily explore its offerings. To accessinformation on the WWW, a user typically utilizes a browser programhaving a graphical interface (such as those offered by such companies asSun, Microsoft, and Netscape) to establish an electronic connectionbetween his/her local client computing system, and a remote serversystem located at an ISP (Internet Service Provider). After suchconnection is made, the user can then perform a number of operationsthrough the browser, including such tasks as uploading/downloading files(including text, graphics, audio, video, etc.) and even executingprograms located on such remote server. The ability to locally executeprograms retrieved from a remote ISP is in fact one of the greaterattractions and promises of the WWW. By having a repository which isaccessible to millions of users simultaneously, program authors have anopportunity to expand the distribution and use of their products to alevel far beyond that previously attainable. To avoid cluttering theuser's local computing system with extraneous program and associatedsupport files, and more importantly so as to provide a measure ofprotection and security to such user, many of such programs are nowbeing implemented using a language known in the art as Java®, and, inparticular, using programming tools known as “applets.” Java® appletsare akin to Java® applications, but the former are specifically designedto interoperate with graphical user interfaces such as the conventionalbrowsers mentioned above. Applets are extremely popular programs todayalso due to the fact that they provide program authors with the tools tocreate multi-media capable programs quickly and easily.

The ease of access to remote programs, however, also increases thepossibility of potential security/privacy breaches at the user's localcomputing system. There is simply no practical method for a user tomonitor the behavior of a remotely retrieved program to ensure that itis not improperly loading data to the user's system, or worse yet,capturing or altering private data from the user's local file system inhis/her computing system. To address this concern, the authors of Java®intentionally constrained applets to operate in what is conventionallyknown as “sand box.” In other words, applets were imbued withsubstantial functionality, but they are not permitted, for example, todo such things as read or write from file systems outside their owndomain (usually the file system of the remote server). So, in the caseof a remotely downloaded applet embodying some code which the userdesires to execute, such applet cannot read or write to the user's localfile system. For a discussion of this limitation of applets, please see“An Introduction to Computer Science—Using Java” by Kamin et. al.,McGraw-Hill, p. 345 (1998). Another limitation, of course, is the factthat an applet cannot make use of data structures (such as graphics fileformats for example) that are incompatible or unreadable with thebrowser within which such applet is executing.

Heretofore this limitation on applets has not posed a substantialbarrier to the use of applet based programs on the WWW, although someattempts have been made to ameliorate the effects of this restriction.For example, some program designers have tried to exploit loopholes inthe sandbox to trick the user's operating system into permitting theapplet to gain local access and print a file on a local printer. Theseprogramming patches, however, are undesirable because they are systemspecific, and are susceptible to being closed down by Java®developers/standards enforcers. Others have suggested relaxing theconstraints on applets which are known by the user to come from averified “clean” source. By requiring an applet to pass through acertification process, some measure of security can be maintained.Again, nonetheless, such “exemption” process is also vulnerable toattack from would-be security invaders, and is therefore unattractive tousers seeking the maximum security intended to be offered by the appletenvironment. It is also inconvenient to users, because they must stillperform the task of evaluating whether a particular program is worthy ofcertification. This, too, reduces the incentives for users to use theWWW, since it requires too much effort for the ordinary user to knowwhat is safe and what is unsafe.

Despite the fact that such efforts have been limited in the past,applicant has realized that the need for a satisfactory solution to theapplet limitation is more crucial now. The inability of an applet toprint to a local printer, for example, means that a local user is unableto capture his/her local input and/or contributions to an applet programdisplaying a remotely retrieved file. This limits the user's enjoymentof the program, since any contributions are lost once the browserprogram is closed. For example, a user who has used an applet in his/herbrowser and accessed a stock price chart located on a remote server, canmake annotations, mark-ups, etc., and see such contributions on adisplay screen. They cannot, however, print a hard copy of such image,and again this reduces significantly the user's enjoyment and theutility of such program.

SUMMARY OF THE INVENTION

An object of the present invention, therefore, is to provide a systemand method for permitting a local resource constrained software routinerunning on a local client system to circumvent such restrictions byexploiting resources at a remote server location;

Another object of the present invention is to provide a system andmethod for permitting a user of a locally executing client internetbrowser to manipulate and perform operations on files containing amixture of local and remote server data;

A related object of the present invention is to provide a system andmethod for permitting a Java® applet to be able to emulate local filetype functionality to a user while still complying with applicablesecurity restrictions on such types of routines when running on a clientmachine;

Yet another object of the present invention is to provide a system andmethod for transferring data between a local computing system and aremote server so that file access privileges on the latter can beexploited by a program running on the former;

A further object of the present invention is to provide a system andmethod for permitting a user to engage in an interactive session duringwhich they may modify content of data files from different remotesources, and still be able to capture and preserve such efforts in alocal file system of the user's computing system;

Another object of the present invention is to provide a system andmethod for a remote server to engage a user to perform the abovementioned objectives.

A data capture program of the present invention is characterizedgenerally by the fact that it is restricted from accessing a first localfile system, but does have access rights to a second file system at aseparate computing system. When program data is generated during aninteractive on-line session that cannot be transferred to such localfile system, the program instead transmits such program data from thelocal computing system to the second file system. The data captureprogram is further configured to interact with a browser on the localcomputing system, which browser has access rights to the local filesystem. In this manner, the browser program can then access the programdata from the second file system, and transfer the program data to anoutput and/or storage device in the local computing system.

In a preferred embodiment, the program data is generally associated withmodified file data resulting from modifications made by a user toinitial data from an initial data file. These modifications couldinclude additions, deletions, edits, etc., of any initial data file typepreferably retrieved from the remote system, such as a file containingchart data, an audio file, a video file, a text file, etc. During theinteractive session involving modifications to chart data, for example,any updates by the user are displayed dynamically in a first window on adisplay device for ease of use. Such modifications, because ofrestrictions on the environment in which such are created, cannotnormally be saved to the user's local file system. Therefore, when theuser wishes to save the results of such modifications, the modified filedata is transferred as noted above to the remote server, where it istreated as described. Thereafter, the modified file data as retrievedfrom the remote server can be displayed in a second window to such userwhere it can be manipulated as desired by the browser program operatingon the user's machine. In this manner, a user can preserve and capturethe results of their efforts during such interactive session on a localcomputing system for future reference.

To transfer the modified file data from the local computing system tothe remote server, an encoder is preferably used to compress the filedata into a standard file format readable by the browser program. In thecase of a visual image data, for example, the preferred approach is toconvert such file into a GIF or JPEG formatted file.

The data capture program of the present invention is preferablyimplemented with one or more Java applets which are in the form of aremote program stored at a remote server. These applets are initiallydownloaded during the interactive session from the remote server butthen execute on the user's local computing system within a restrictedenvironment within the user's browser program. The remote program ispreferably configured such that it interacts with the browser programand performs at least the following operations:

(1) retrieving an initial data file from the remote server for useduring the interactive session; and

(2) displaying information relating to the initial data file in adisplay window visible to the user; and

(3) accepting modifications to such initial data file during theinteractive session; and

(4) saving such modifications to such initial data file in an modifieddata file;

(5) routing the modified data file to the remote server; and

In this fashion, the modified data file can be retrieved by the user ata later time and saved to the local file system by the browser programas noted above.

To further assist the user, the remote program also provides usercommand functions in the display window, which command functions can beused to generate the modifications more easily. As the modifications aremade, the user is preferably given feedback to confirm the same (such aswith a line drawn on a chart for example). The invention is alsoflexible in that more than one initial data file can be loaded from oneor more remote servers and modified, thus permitting essentiallyunlimited editing capability.

A remote server of the present invention is configured to permit a userremote from such server to engage in an interactive on-line session withsuch server using the aforementioned remote program. In this way, theserver can interact with the local machine as described above.

In a preferred embodiment of the remote program permitting a user toannotate chart data with visual labels and descriptions, an interactiveportion of such program has a first executable routine for processinginitial data from the initial data file obtained from the remote serverand for storing first image data associated with the initial data in awindow image data buffer. The browser program then causes the localcomputing system to display the first image data in a window on adisplay to the user. A second executable routine generates modified databased on the user's modifying of the initial data with the visual labelsand descriptions. A third routine then generates an modified data filebased on the modified data, and then further transmits the modified datafile to the remote server. Thereafter, the first routine, in cooperationwith the browser, can also process the modified data file from theremote server, and thus the browser program can communicate the modifieddata file to a file system in the local computing system.

An interactive session between a local computing system and a remoteserver is therefore conducted in accordance with the present inventionusing the following preferred steps:

-   -   (a) accessing a remote program located on the remote server        using a browser program located on the local computing system;    -   (b) executing the remote program on the local computing system;    -   (c) accessing an initial data file from the remote server with        the remote program for use during the interactive session; and    -   (d) displaying an initial display image based on initial data        from the initial data file; and    -   (e) modifying the initial data;    -   (f) saving the modified initial data as modified data; and    -   (g) transmitting an modified data file corresponding to the        modified data from the local computing system to the server; and    -   (h) accessing the modified data file from the remote server with        the remote program;    -   (i) communicating the modified data file to a file system in the        local computing system using the browser program.

In the above described method, the modified data is derived from aninitial data file (which may include more than one initial data file)and supplemental data input under control of a user of the remoteprogram.

In a further variation, the updated data file is image data compressedusing an encoder which translates a pixel stream into a file formatusable by the browser program or another program having access to thelocal file system.

Data transfers of the present invention are accomplished through anon-line connection between a local computing system and a remote serverpreferably using the following steps:

-   -   (a) executing an interactive program on the local computing        system, which interactive program coordinates with an on-line        connection management program on such local computing system,        and which interactive program further is restricted from        accessing a local file system on such local computing system;    -   (b) accessing initial data information for use on the local        computing system; and    -   (c) generating modified data information, which modified data        information includes the initial data information and any        additional data supplemental information added under control of        a user of the interactive program; and    -   (d) saving the modified data information;    -   (e) transmitting the modified data information from the local        computing system to the server; and    -   (f) accessing the modified data information from the remote        server;    -   (g) communicating the modified data information to an output        and/or storage device in the local computing system using the        on-line connection management program.

Again, in a preferred embodiment, the modified data is compressed imagedata derived from dynamically modifying an initial data file. This imagedata is compressed using an encoder which translates a pixel stream intoa file format usable by the second program. Nonetheless, the initialdata file can be in a graphics file format, an audio file format, a textformat, a video format, or some combination thereof.

An interactive, on-line session of the present invention permitting auser to engage in an interactive on-line session with a server using aremote program downloaded from the server but executing on the user'slocal computing system, and wherein the remote program is restrictedfrom accessing a file system on such local computing system, isaccomplished as follows:

-   -   (a) coordinating communications between the server and a browser        program also executing on the user's local computing system        during the interactive on-line session, the browser program        further including software routines for interacting with the        file system on the user's local computing system; and    -   (b) receiving data at the server consisting of modified        information transmitted from the remote program;    -   (c) creating an modified file for the modified information at        the server, which modified file is accessible by the remote        program;    -   (d) transferring the modified file to the local computing system        so that the browser program can utilize the modified file;

In this manner, the browser program can thereafter transfer suchmodified file to an output and/or storage device in the local computingsystem, such as a printer, a local file system, etc.

Although the inventions are described below in a preferred embodiment,it will be apparent to those skilled in the art the present inventionwould be beneficially used in many environments where it is necessary toprovide security constrained software routines with additionalfunctionality.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an embodiment of a systemimplemented in accordance with the teachings of the present invention;

FIG. 2 depicts in flow chart form the general method of operation of thesystem noted in FIG. 1;

FIGS. 3A to 3C illustrates the output generated by the present systemand method as such would be seen on a display device by a user of suchsystem;

DETAILED DESCRIPTION OF THE INVENTION

A preferred embodiment of a system 100 constructed in accordance withthe present disclosure is shown in block diagram form in FIG. 1. A localclient computing system 102 includes a number of components found in anytypical contemporary computing system, including a processor 105operable with a system code/data memory 110 (normally a DRAM), one ormore input devices 115 (usually a keyboard, mouse, joystick or thelike), a display memory 120 and associated buffer 121) (typically asupplemental fast VRAM or DRAM separate from system memory 110), adisplay 125 (preferably a CRT monitor, LCD panel etc.), one or morestorage devices 130 (ordinarily a hard disk drive or othermagnetic/optical based devices), an output device 135 (usually aprinter) and an I/O interface 140 (preferably a high speed modem andassociated drivers) for establishing an electronic connection 142 to theinternet, an intranet, or other computer network. All of theaforementioned components are conventional, and it is understood thatsuch computing system may range in complexity spanning the spectrum froma large multi-user mainframe down to personal electronic assistants.

Operating within code/data memory 110 under control of processor 105 isa browser program 150, and an applet 160 executing within such browser.This applet runs in a standard Java virtual machine (JVM) within thebrowser. This operational relationship is denoted by browser program 150enclosing applet 160; it will be apparent to those skilled in the artthat this visual representation is merely intended for illustrativepurposes, and does not purport to describe the entirety of therelationship between the browser and the applet. Moreover, since thisrelationship is well-known in the art, see e.g., the Kamins referencediscussed above, it is not necessary to replicate such details here.Browser program 150 (preferably one of the more robust versions known inthe art, such as Netscape Navigator or Microsoft Explorer) is configuredto operate within system memory 110, and is capable ofinteracting/communicating of course with the other components withinsystem 102 in conventional fashion, as well as through internet link 142with a remote server 180 located at an ISP site. Remote server 180 alsoincludes many of the same components of system 102 (albeit in largerscale to accommodate the needs of several potential data requesters)such as an I/O interface 181, a processor 182, a server code/data memory183, and a storage system 185. This system is used to store both aninitial data file 186, and an modified data file 187, which items arediscussed in more detail below.

Returning back to applet routine 160, it can be seen in FIG. 1 that thisitem includes applet code 162, which is written in Java® programmingcode; a listing of the source code used in the present invention isappended to the end of the present disclosure as Appendix A. This Java®code can be executed by means of a Java® interpreter in browser 160 tocarry out operations within system 102. Applet code 162 has access to anapplet data buffer 164, which, as shown in FIG. 1, is designed to storeincoming/outgoing data through internet link 142. As seen in FIG. 1,initial window data 165, as well as updated window data 166 are locatedand stored within such applet data buffer 164. Finally, applet code 162also has access to an associated applet window data buffer 166, which itcan use for painting a window on display 125 as explained further below.This buffer can be generated with conventional Java® programming tools,including, for example, the createlrnage( ) command to any desired sizefor the applet and application in question. The relationship between thevarious applet routines and resources can be gleaned further from thesource code listing attached hereto, which includes all of the routinesdiscussed herein except for the freeware programs noted below.

With the exception of the above details pertaining to applet 160, theother circuitry, structures and routines embodied in the block diagramof FIG. 1 are not material to the teachings of the present invention.These items are well-known in the art, and can be implemented by skilledartisans in a variety of ways. Moreover, it will be apparent to those ofskill in the art that a number of conventional (but extraneous in thiscase) features of client system 102 and server system 180 have beenomitted to better illustrate the present teachings.

As FIG. 1 suggests, to maximize security and privacy for the user'slocal client computing system 102, applet 160 is operationallyrestricted and has only limited access to components within system 102.It cannot, for example, directly interact with a storage device 135, oran output device 135 in order to create (or read) a file within localclient system 102. Nor can applet 160 operate to transfer a filedirectly to browser program 160. This is an undesirable situation,since, as mentioned above, it is likely that many users in the futurewill download files from remote servers, modify such files, and yet notbe able to do anything useful with them afterwards. For example, theywill not be able to save such modifications for use with other programson the user's computing system. It will be understood by those skilledin the art that Java language applets are only one form of programmingwhich may be subject to such restrictions, and that the presentinvention is by no means limited to such specific contexts.

To overcome this limitation, applet code 162 of the present inventionincludes instructions which permit the local client computing system 102of FIG. 1 to carry out the method of the present invention as shown inflow chart form in FIG. 2. At step 205, a user initiates an onlinesession from his/her local client computing system 102 in conventionalfashion, such as by dial-up or always-on access to an ISP. Browser 160is then activated at step 210 to permit the user to see, interact andaccess other programs, files, etc. at step 215; such items may belocated on the internet, including at a WWW compatible remote server 180through an http link or the like. When the user desires to interact witha remote program on remote server 180, one or more applets 160 necessaryfor operating such program will be transmitted downstream to the user'scomputing system 102 through link 142 at step 220 where they will beloaded into a code/data memory 110 for execution on local computingsystem 102 during step 225. In a preferred embodiment, the remoteprogram accessed and loaded by the user is located at a websitemaintained by Prophet Information Services at www.prophetcharts.com.This program is known as “Prophet Charts,” and generally includes suchfunctionality as: (1) permitting a user to retrieve historical stockdata from a file stored on a remote server; (2) placing such stock datain graphical form in a visible display window at the user's end; (3)permitting the user to annotate the chart with trend lines, labels, etc.to glean trading patterns, potential buy/sell opportunities, etc.Accordingly, in the event the user requests to see data associated withan initial remote data file 186 (FIG. 1)—as for example, in the case ofa historical chart showing the price of a particular stock for aparticular period of time this file is retrieved by applet code 162 atstep 230. In this instance, as described above, applet 160 can accesssuch data in file 186 because the latter is located logically in thesame environment from which applets 160 are derived. To ensure that suchdata is usable by the user within system 102, file 186 must be retrievedfrom server 180 in a format that is readable or decodable by browser160; i.e., in text form, or some graphics form that is acceptable, suchas GIF, TIFF, JPEG, or the like, depending on the particular browserprogram 160.

In any event, initial data from file 186 is stored within Applet DataBuffer 164 at location 165 (as shown in FIG. 1) where it is processed byapplet code 162 and then placed into Applet window data buffer 166. Bycoordinating with browser 160, a graphical representation of initialdata file 186 is then passed through display buffer 121, where it thenis placed in display memory 120 as noted in the dotted lines shown inFIG. 1. Subsequently, the applet window image data then also becomesvisible to the user on display 125 as noted in step 235 of FIG. 2.

Up until this point, the above description has merely described thetypical operation of any prior art Java® applet program loaded from aremote site onto a local client computing system. In the presentembodiment, nonetheless, a number of additional unique operative stepsare implemented to provide such program user with functionality notpreviously available.

Referring again to FIG. 2, the primary features, benefits, etc. of thepresent invention are based in large part on the operations carried outby applet code 162 beginning at step 240. At that point in the routine,the user is permitted to interact with the data presented in the appletwindow image on display 125. In particular, in a preferred embodiment,the user is allowed to add annotations to a graphical chart painted insuch window. These annotations could include for example, data enteredby the user manually from a keyboard/mouse 115, additional datagenerated automatically by applet code 162 in response to a user commandfrom such input device, or some other authorized and accessible datasource for the applet. These modifications (including deletions,supplementations or embellishments) of the window image data are handledat step 245 until the user is finished and wants to do some other task(i.e., saving the modified image data, or perhaps working with anotherimage). Again, with reference to FIG. 1, it is apparent that suchadditional data entered at step 240 can be passed through to applet 160and processed by applet code 162 to update applet window data buffer164. From there it is passed on to display buffer 121 before it is usedto update display memory 120 and the user's display 125. Nonetheless,even though the window image data has been updated to include theefforts of the user to include such extra user generated data, it is noteasily “capturable,” so as to help the user preserve their effortsbecause as it is stored it is not in a format that is understandable bybrowser program, and cannot be output by applet 160 to a storage device130 because of the aforementioned security restrictions.

Returning to FIG. 2, the present invention permits a user to retain theresults of their efforts, and capture their modifications to the initialdata in the following manner. First, at step 250, the user is promptedto determine if they would like to capture and preserve the modifiedwindow image data, i.e., such as by printing in hard copy form, orstoring in some electronic form. If not, the user can retrieve adifferent file at step 230 or perform another operation of theirchoosing. Should the user desire to capture the updated window image,however, applet code 162 then proceeds at step 255 to convert suchwindow image data into modified window data, which is stored in location166 (FIG. 1). First, the window image data is captured using a built-instandard Java® programming tool known as pixelGrabber. This routinecaptures the pixels from the Java® image and renders them into aone-dimensional array of integers. Each integer contains the red greenand blue components of the color of that pixel. Then, this array ispassed through a converter, where it is translated or converted into astandard graphics file format. This conversion is preferably performedby a pixel-to-gif converter to convert the pixels of the applet windowimage data into a Graphics Interchange Format (GIF) file. Thisconversion can be accomplished, for example, by a number of prior artprograms, including by a freeware software routine authored by JefPoskanzer entitled GifEncoderjava and available at any number of filesites on the Internet including at www.acme.com. It should be noted thatthe selection of the particular file format, and/or particular converteris not material to the present teachings, and the present invention isby no means restricted to any particular combination thereof. The onlyimportant parameter, as explained below, is that the modified appletwindow image data must be changed into a format that is compatible withbrowser program 160 as noted below. Alternatively, such format need onlybe readable by browser program 160 so that it can be downloaded and usedby another program on the local computing system 102.

In any event, after such conversion, at step 260 modified window data166 is then uploaded by applet 160 to remote server 180, where it can bewritten to remote data storage device 185 in a location 187 as amodified data file. This happens by opening a connection to aprogramming script running on web server 180 at the site the user isconnected to. In the preferred embodiment, this is a script isimplemented in “perl” which is a conventional, well-known scriptinglanguage commonly used by those skilled in the art. Basically, the perlscript running on server 180 merely fetches the next available imagenumber and writes the data from modified window data file 166(containing the compressed array) to it, with a designation such aschartxxx.gif, where xxx is the next available number available on server180. Because this file is in compressed format, the transfer timebetween local computing system 102 and remote server 180 should berelatively reasonable. Furthermore, it is expected that improvements inhigh speed internet access will likely obviate any concerns about suchtypes of relatively small scale data transfers.

At this point the captured user modified image data is now located on aserver which can be accessed by browser program 150. The perl script onserver 180, therefore, replies to applet 160 at step 265 with a URLindicating the location of the new file, such as“www.prophetcharts.corn/graphs/chartXXX.gif” and applet 160 then opens anew browser window, which can be displayed to the user on display 125,which points to that image file. Because such file is also retrieved ina format usable by browser 160, it can be manipulated and treated likeany other file so that, for example, at step 270 it can now be saved inelectronic form on local storage device 130, or printed on a localoutput device 135. The user can then proceed to retrieve additional datafiles at step 230, or as noted above, simply go on to another operation.

As a housekeeping matter, and to save storage space, any such additionalfiles stored on server 180 can be deleted after the user's session isover. It can be seen from the above description, therefore, that aserver 180 is configured to permit a user remote from such server toengage in an interactive on-line session with such server using theaforementioned applet 160. In this way, server 180 can interact andexchange data with local computing system 102 as described above, andpreserve any contributions made during such session for the user'sbenefit.

While the above process appears somewhat circuitous, applicants haveconfirmed that it in fact proceeds very rapidly, and from theperspective of the user, does not involve an inordinate delay.Furthermore this process retains the integrity of the intended frameworkfor Java® applets by ensuring that they do not access local resources.In this manner, users are guaranteed that the applets are kept in theJava® “sandbox,” and not permitted to perform operations that mightcompromise the user's privacy or security The present invention isextremely useful, therefore, for preserving and capturing the user'sedits, modification, supplementations of an applet window's image dataduring an interactive session with a remote server, which, absent thepresent teachings, would be otherwise lost resulting in reducedproductivity, utility, user satisfaction, etc. It is expected that thepresent teachings can be used in any number of Java® applet basedprograms, therefore, where it is desirable to permit the user tointeract with applet data and yet still permit the user to exploit theuse of local resources without the restrictions normally imposed onapplet code. The present invention opens up a new realm of possibilitiesfor WWW programmers, web-site operators, on-line vendors and users ofthe since it now allows data to be freely transferred and then modifiedunder the user's control in a manner that preserves the user'scontributions.

FIGS. 3A to 3C are a series of screen shots illustrating the operationof the present invention as perceived by a user of a remote programwhich is used to launch locally running applets on the user's localcomputing system. In FIG. 3A, a display 300 presents a user with agraphical interface 310 (generated by underlying browser 160) and anapplet generated window image 320, which window contains applet windowimage data 325 (derived from initial data file 165) as well as commandbuttons/activators 328 at the top of window 320. The latter are used, asexplained above, to permit a user to interact with the image presentedin window 320, so that, as seen in FIG. 3B, such user can add additionalinformation to such window data, remove information from the window,modify the existing data, etc. In this instance, the user has added atrend line 330 notation to the chart data image to reflect someadditional perception by such user about such data that is notrepresented in the original data file 166; for example, an indicatorshowing the general trend of a stock price over time. Other image dataannotations are of course possible, including descriptive labels, appletgenerated calculations, etc. A second data file containing data for asecond stock could be subsequently loaded, for example, so that acomposite image having data from both the first and second stocks couldbe seen in window 320.

Assuming the user desires to capture the image data including the addedtrend line 330, the conversion steps described above beginning with step255 (FIG. 2) are then performed as noted. Looking at FIG. 3C, thereturned modified data file 166 is then displayed in a graphics imageformat in a separate window 350 where, as noted earlier, it can beprinted, saved, etc. by the user.

As alluded to briefly above, in another variation of the presentinvention, applet 160 can also permit such user to import an additionaldata file 166′ from server 180, for example, representing data for adifferent stock so that a combined data image utilizing data from twodifferent files 166 and 166′ can be observed simultaneously in anoverlay fashion and manipulated by the user. Such data from thedifferent files can be represented in different colors in applet dataimage window 320 to help the user differentiate the behavior of the twosuperimposed data sets for the selected two stocks. Such combined datafrom the two files (or more if desired) can then be converted into asuitable file format as explained above for the single image case, andthen handled subsequently in the same manner.

In yet another embodiment, the modified data file can includeinformation other than graphics data. For example, the user can downloada standard audio file from the remote serve, edit such audio file usingany conventional audio file editor, and then save any modifications inthe same manner as performed above for the image data. Then, instead ofreceiving back a GIP file, a WAV file, MIDI file, etc., could bereturned by the server. Standard encoders for compressing audioinformation can be used for generating the modified file at the localcomputing system before sending on this file to the server for laterretrieval. Similar editing sessions with other types of files retrievedduring an on-line session and manipulated by the user could be capturedas well. In this manner, a user's contributions associated with an audiofile, a text file, a video file, or any kind of multimedia file can bepreserved and stored on the local computing system, even when usingrestricted code such as Java applets.

Although the present invention has been described in terms of apreferred embodiment, it will be apparent to those skilled in the artthat many alterations and modifications may be made to such embodimentswithout departing from the teachings of the present invention.Accordingly, it is intended that the all such alterations andmodifications be included within the scope and spirit of the inventionas defined by the following claims.

Attached hereto is an appendix of the source code used in the variousapplet routines described above.

1. A method of generating a Java-based chart comprising the steps of:retrieving chart data using a hypertext transfer protocol from a remoteworld-wide web accessible server system using a Java-based appletprogram executing within a window of a browser program on a clientcomputing system; wherein said chart data relates to a price history fora company stock; presenting said chart data in graphical form in saidwindow for viewing by a user as a visual historical price chart for saidcompany stock; wherein said visual historical price chart is generatedusing display data from a window data buffer used by said Java-basedapplet program; providing a set of user selectable commands within saidwindow for permitting said user to annotate said visual historical pricechart directly at said client computing system, including with one ormore annotation items; wherein said annotation items include at leastone of the following for said visual historical price chart: trend linesand/or descriptive text labels provided by said user; modifying saiddisplay data directly at said client computing system in said windowdata buffer based on said user annotating said historical chart with oneor more of said annotation data items to generate modified display data;wherein said user can see both said visual historical price chart aswell as such user-supplied annotation items in said window of saidbrowser program using said modified display data.
 2. The method of claim1, wherein said user can also alter an appearance of said window byremoving data from said window.
 3. The method of claim 1, wherein saiduser can present chart data for more than one company stock within saidwindow, and such that different colors are used for displaying visualhistorical price charts for different company stocks.
 4. The method ofclaim 1, wherein said modified display data can be captured in permanentform by transmitting the same to the remote server.
 5. The method ofclaim 1, wherein said user can provide said annotation items using acomputer mouse or a keyboard coupled to said client computing system. 6.The method of claim 1 wherein said Java-based applet program executingwithin a window of a browser program on the client computing system hasaccess to I/O and/or storage resources of said remote world-wide webaccessible server system, but is restricted from accessing a local filesystem on the client computing system.
 7. The method of claim 1 andfurther comprising: transmitting said modified display data to theremote world-wide web accessible server system; and using the Java-basedapplet program executing within a window of the browser program on theclient computing system to obtain the modified display data.
 8. Themethod of claim 7 wherein said using comprises using the Java-basedapplet program executing within a window of the browser program on theclient computing system to obtain the modified display data for storagein a location on the client computing system that is accessible by theJava-based applet program for printing and storage.
 9. The method ofclaim 1 wherein the Java-based Applet program is executing within abrowser program on a local file system of the client computing system,said Java-based applet program having access to I/O and/or storageresources of said remote world-wide web accessible server system andrestricted access to said local file system of the client computingsystem, the method further comprising: transmitting said modifieddisplay data to the remote world-wide web accessible server system; andobtaining the modified display data from the remote world-wide webaccessible server system to allow the Java-based applet program on thebrowser program of the client computing system to print and locallystore the modified display data including the user-supplied annotationitems.
 10. An apparatus to generate a Java-based chart comprising: aclient computing system, the client computing system including a browserprogram; the client computing system adapted to: retrieve chart datausing a hypertext transfer protocol from a remote world-wide webaccessible server system using a Java-based applet program executingwithin a window of the browser program, said chart data relating to aprice history for a company stock; present said chart data in graphicalform in said window for viewing by a user as a visual historical pricechart for said company stock, said visual historical price chart to begenerated using display data from a window data buffer used by saidJava-based applet program; provide a set of user selectable commandswithin said window for permitting said user to annotate said visualhistorical price chart directly at said client computing system,including with one or more annotation items, said annotation itemsinclude at least one of the following for said visual historical pricechart: trend lines and/or descriptive text labels provided by said user;and modify said display data directly at said client computing system insaid window data buffer based on said user annotating said historicalchart with one or more of said annotation data items to generatemodified display data, wherein said user can see both said visualhistorical price chart as well as such user-supplied annotation items insaid window of said browser program using said modified display data.